Lifelong Variational Autoencoder via Online Adversarial Expansion Strategy

The Variational Autoencoder (VAE) suffers from a significant loss of information when trained on non-stationary data distribution. This in VAE models, called catastrophic forgetting, has not been studied theoretically before. We analyse the forgetting behaviour continual generative modelling by developing new lower bound likelihood, which interprets process as an increase probability distance between generator's distribution and evolved proposed shows that VAE-based dynamic expansion model can achieve better performance if its capacity increases appropriately considering shift Based this analysis, we propose novel criterion aims to preserve diversity among components, while ensuring it acquires more knowledge with fewer parameters. Specifically, implement perspective multi-player game Online Adversarial Expansion Strategy (OAES), considers all previously learned components well currently updated component multiple players game, adversary evaluates their performance. OAES dynamically estimate discrepancy each player without accessing task information. leads gradual addition them. show empirically extension strategy enable best given appropriate size.